Drop cache_loaded_ flag; gate init on metadata.empty() (#20245)#20245
Conversation
🔗 Helpful Links🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/20245
Note: Links to docs will display an error until the docs builds have been completed. ❌ 4 New Failures, 2 Unrelated FailuresAs of commit 3d70e60 with merge base 0632135 ( NEW FAILURES - The following jobs have failed:
FLAKY - The following job failed but was likely due to flakiness present on trunk:
BROKEN TRUNK - The following job failed but was present on the merge base:👉 Rebase onto the `viable/strict` branch to avoid these failures
This comment was automatically generated by Dr. CI and updates every 15 minutes. |
|
@doggeral has exported this pull request. If you are a Meta employee, you can view the originating Diff in D108178967. |
This PR needs a
|
Summary: Follow-up to D106717093. The cache_loaded_ flag was used as the gate in initialize_for_runtime to decide between the CACHE_LOADED fast-path (reopen fd only) and the load-or-fresh-write slow path. But it was only set true by load_packed_cache(), not by the fresh-write path. After a fresh-write + save_packed_index closed the fd, the next init found cache_loaded_=false → re-entered load_packed_cache → push a second whole-file mmap region on top of the per-entry regions from reserve_space. Same file mapped twice; redundant VM and bookkeeping. Replace the flag with !name_to_packed_data_metadata_.empty() as the gate. Metadata is non-empty after BOTH paths (load and fresh-write), so the gate is correct for both. Removes 1 field, 3 writes, and the latent double-mmap on the fresh-write-then-save path. The from_load entry comment is updated to drop the now-stale reference to cache_loaded_ auto-invalidation. Differential Revision: D108178967
7889e6b to
a4578ad
Compare
Summary: Follow-up to D106717093. The cache_loaded_ flag was used as the gate in initialize_for_runtime to decide between the CACHE_LOADED fast-path (reopen fd only) and the load-or-fresh-write slow path. But it was only set true by load_packed_cache(), not by the fresh-write path. After a fresh-write + save_packed_index closed the fd, the next init found cache_loaded_=false → re-entered load_packed_cache → push a second whole-file mmap region on top of the per-entry regions from reserve_space. Same file mapped twice; redundant VM and bookkeeping. Replace the flag with !name_to_packed_data_metadata_.empty() as the gate. Metadata is non-empty after BOTH paths (load and fresh-write), so the gate is correct for both. Removes 1 field, 3 writes, and the latent double-mmap on the fresh-write-then-save path. The from_load entry comment is updated to drop the now-stale reference to cache_loaded_ auto-invalidation. Differential Revision: D108178967
Summary: Follow-up to D106717093. The cache_loaded_ flag was used as the gate in initialize_for_runtime to decide between the CACHE_LOADED fast-path (reopen fd only) and the load-or-fresh-write slow path. But it was only set true by load_packed_cache(), not by the fresh-write path. After a fresh-write + save_packed_index closed the fd, the next init found cache_loaded_=false → re-entered load_packed_cache → push a second whole-file mmap region on top of the per-entry regions from reserve_space. Same file mapped twice; redundant VM and bookkeeping. Replace the flag with !name_to_packed_data_metadata_.empty() as the gate. Metadata is non-empty after BOTH paths (load and fresh-write), so the gate is correct for both. Removes 1 field, 3 writes, and the latent double-mmap on the fresh-write-then-save path. The from_load entry comment is updated to drop the now-stale reference to cache_loaded_ auto-invalidation. Differential Revision: D108178967
Summary: Follow-up to D106717093. The cache_loaded_ flag was used as the gate in initialize_for_runtime to decide between the CACHE_LOADED fast-path (reopen fd only) and the load-or-fresh-write slow path. But it was only set true by load_packed_cache(), not by the fresh-write path. After a fresh-write + save_packed_index closed the fd, the next init found cache_loaded_=false → re-entered load_packed_cache → push a second whole-file mmap region on top of the per-entry regions from reserve_space. Same file mapped twice; redundant VM and bookkeeping. Replace the flag with !name_to_packed_data_metadata_.empty() as the gate. Metadata is non-empty after BOTH paths (load and fresh-write), so the gate is correct for both. Removes 1 field, 3 writes, and the latent double-mmap on the fresh-write-then-save path. The from_load entry comment is updated to drop the now-stale reference to cache_loaded_ auto-invalidation. Differential Revision: D108178967
Summary: Follow-up to D106717093. The cache_loaded_ flag was used as the gate in initialize_for_runtime to decide between the CACHE_LOADED fast-path (reopen fd only) and the load-or-fresh-write slow path. But it was only set true by load_packed_cache(), not by the fresh-write path. After a fresh-write + save_packed_index closed the fd, the next init found cache_loaded_=false → re-entered load_packed_cache → push a second whole-file mmap region on top of the per-entry regions from reserve_space. Same file mapped twice; redundant VM and bookkeeping. Replace the flag with !name_to_packed_data_metadata_.empty() as the gate. Metadata is non-empty after BOTH paths (load and fresh-write), so the gate is correct for both. Removes 1 field, 3 writes, and the latent double-mmap on the fresh-write-then-save path. The from_load entry comment is updated to drop the now-stale reference to cache_loaded_ auto-invalidation. Reviewed By: GregoryComer Differential Revision: D108178967
a4578ad to
3a7dca4
Compare
Summary: Follow-up to D106717093. The cache_loaded_ flag was used as the gate in initialize_for_runtime to decide between the CACHE_LOADED fast-path (reopen fd only) and the load-or-fresh-write slow path. But it was only set true by load_packed_cache(), not by the fresh-write path. After a fresh-write + save_packed_index closed the fd, the next init found cache_loaded_=false → re-entered load_packed_cache → push a second whole-file mmap region on top of the per-entry regions from reserve_space. Same file mapped twice; redundant VM and bookkeeping. Replace the flag with !name_to_packed_data_metadata_.empty() as the gate. Metadata is non-empty after BOTH paths (load and fresh-write), so the gate is correct for both. Removes 1 field, 3 writes, and the latent double-mmap on the fresh-write-then-save path. The from_load entry comment is updated to drop the now-stale reference to cache_loaded_ auto-invalidation. Reviewed By: GregoryComer Differential Revision: D108178967
3a7dca4 to
3d70e60
Compare
Summary: Follow-up to D106717093. The cache_loaded_ flag was used as the gate in initialize_for_runtime to decide between the CACHE_LOADED fast-path (reopen fd only) and the load-or-fresh-write slow path. But it was only set true by load_packed_cache(), not by the fresh-write path. After a fresh-write + save_packed_index closed the fd, the next init found cache_loaded_=false → re-entered load_packed_cache → push a second whole-file mmap region on top of the per-entry regions from reserve_space. Same file mapped twice; redundant VM and bookkeeping. Replace the flag with !name_to_packed_data_metadata_.empty() as the gate. Metadata is non-empty after BOTH paths (load and fresh-write), so the gate is correct for both. Removes 1 field, 3 writes, and the latent double-mmap on the fresh-write-then-save path. The from_load entry comment is updated to drop the now-stale reference to cache_loaded_ auto-invalidation. Reviewed By: GregoryComer Differential Revision: D108178967
Summary:
Follow-up to D106717093. The cache_loaded_ flag was used as the gate
in initialize_for_runtime to decide between the CACHE_LOADED
fast-path (reopen fd only) and the load-or-fresh-write slow path.
But it was only set true by load_packed_cache(), not by the
fresh-write path. After a fresh-write + save_packed_index closed
the fd, the next init found cache_loaded_=false → re-entered
load_packed_cache → push a second whole-file mmap region on top
of the per-entry regions from reserve_space. Same file mapped
twice; redundant VM and bookkeeping.
Replace the flag with !name_to_packed_data_metadata_.empty() as
the gate. Metadata is non-empty after BOTH paths (load and
fresh-write), so the gate is correct for both. Removes 1 field,
3 writes, and the latent double-mmap on the fresh-write-then-save
path. The from_load entry comment is updated to drop the
now-stale reference to cache_loaded_ auto-invalidation.
Reviewed By: GregoryComer
Differential Revision: D108178967